This notebook contains a set of analyses for analyzing GOBBluth89’s BoardGameGeek collection. The bulk of the analysis is focused on building a user-specific predictive model to predict the games that the specified user is likely to add to their collection.
By analyzing a user’s collection and training a predictive model, I am able to answer questions such as:
What designers/mechanics/genres does a user tend to like or dislike?
What older games might they be interested in adding to their collection?
What new and upcoming games should they check out?
How many games has GOBBluth89 owned/rated/played?
What types of game does GOBBluth89 own? I can look at the most frequent types of categories, mechanics, designers, and artists that appear in a user’s collection.
What games does GOBBluth89 currently have in their collection? The following table can be used to examine games the user owns, along with some helpful information for selecting the right game for a game night!
Use the filters above the table to sort/filter based on information about the game, such as year published, recommended player counts, or playing time.
I’ll now the examine predictive models trained on the user’s collection.
For an individual user, I train a predictive model on their collection in order to predict whether a user owns a game. The outcome, in this case, is binary: does the user have a game listed in their collection or not? This is the setting for training a classification model, where the model aims to learn the probability that a user will add a game to their collection based on its observable features.
How does a model learn what a user is likely to own? The training process is a matter of examining historical games and finding patterns that exist between game features (designers, mechanics, playing time, etc) and games in the user’s collection.
Note: I train models to predict whether a user owns a game based only on information that could be observed about the game at its release: playing time, player count, mechanics, categories, genres, and selected designers, artists, and publishers. I do not make use of BGG community information, such as its average rating or number of user ratings (though I do use a game’s estimated complexity as a feature). This is to ensure the model can predict newly released games and is not dependent on the BGG community to rate them.
A predictive model gives us more than just predictions. We can also ask, what did the model learn from the data? What predicts the outcome? In the case of predicting a boardgame collection, what did the model find to be predictive of games a user owns?
To answer this, I can examine the coefficients from a model logistic regression with ridge regularization (which I will refer to as a penalized logistic regression). Positive values indicate that a feature increases a user’s probability of owning/rating a game, while negative values indicate a feature decreases the probability. To be precise, the coefficients indicate the effect of a particular feature on the log-odds of a user owning a game.
This model examines a wide variety of features of games (506 features, to be exact) and estimates their effect on whether a user owns a game. These estimates are then shrunken towards zero based on a tuning parameter (lambda), where the appropriate value is estimated from the data.
The following visualization shows the path of each feature as it enters the model, with highly influential features tending to enter the model early with large positive or negative effects.
This type of model enables me to I can examine the effects of specific features on a user’s collection. For instance, what is a user’s favorite designer? Least favorite mechanic? The following plots indicate specific effects for different kinds of features.
In addition to training a logistic regression, I trained another type of model using boosted trees (LightGBM), a flexible nonparametric method that is well suited for prediction.
Which features were most used by this model? Features that are important in predicting a user’s collection will appear towards the top of cover, frequency, and/or gain.
How well did the model do in predicting the user’s collection?
This section contains a variety of visualizations and metrics for assessing the performance of the model(s). If you’re not particularly interested in predictive modeling, skip down further to the predictions from the model.
An easy way to examine the performance of classification model is to view a separation plot.
I plot the predicted probabilities from the model for every game (from resampling) from lowest to highest. We then overlay a blue line for any game that the user does own. A good classifier is one that is able to separate the blue (games owned by the user) from the white (games not owned by the user), with most of the blue occurring at the highest probabilities (right side of the chart).
I can more formally assess how well each model did in resampling by looking at the area under the receiver operating characteristic curve (roc_auc). A perfect model would receive a score of 1, while a model that cannot predict the outcome will default to a score of 0.5. The extent to which something is a good score depends on the setting, but generally anything in the .8 to .9 range is very good while the .7 to .8 range is perfectly acceptable.
| type | wflow_id | .metric | mean | std_err | n |
|---|---|---|---|---|---|
| resamples | glmnet | roc_auc | 0.914 | 0.007 | 5 |
| resamples | lightgbm | roc_auc | 0.896 | 0.022 | 5 |
Another way of looking at what the model learned is to see its predictions on the training set. The models are trained on games published before 2021; of these games, what did the model like for the user?
| Top (Older) Games for GOBBluth89 | |||||
| Rankings based on predictive model trained on user's collection using games released through 2021 | |||||
| rank | image | game | description | Pr(Own) | Own |
|---|---|---|---|---|---|
| 1 | Star Wars: Rebellion (2016) | Star Wars: Rebellion is a board game of epic conflict between the Galactic Empire and Rebel Alliance for two to four players. Experience the Galactic Civil War like never before. In Rebellion, you control the entire Galactic Empire or the fledgling Rebel Alliance. You must command starships, account for troop movements, and rally systems to your cause. Given the differences between the Empire ... | 1.000 | no | |
| 2 | Unmatched: Battle of Legends, Volume Two (2021) | In battle, there are no equals. Unmatched is a highly asymmetrical miniature fighting game for two or four players. Each hero is represented by a unique deck designed to evoke their style and legend. Tactical movement and no-luck combat resolution create a unique play experience that rewards expertise, but just when you've mastered one set, new heroes arrive to provide all new match-ups. Batt... | 0.989 | yes | |
| 3 | Unmatched: Jurassic Park – InGen vs Raptors (2020) | In battle, there are no equals. "They show extreme intelligence, even problem-solving intelligence. Especially the big one." He has hunted many a dangerous predator, but against a pack of raptors, is Robert Muldoon the hunter or the hunted? In Unmatched: Jurassic Park – InGen vs Raptors, these clever girls use their speed and agility to surround their prey, getting stronger when they attack to... | 0.973 | yes | |
| 4 | Citadels (2000) | In Citadels, players take on new roles each round to represent characters they hire in order to help them acquire gold and erect buildings. The game ends at the close of a round in which a player erects his/her eighth building. Players then tally their points, and the player with the highest score wins. Players start with a number of building cards in their hand; buildings come in five colors,... | 0.968 | no | |
| 5 | Unmatched: Cobble & Fog (2020) | In battle, there are no equals. Unmatched: Cobble & Fog features four new heroes for the Unmatched system. Invisible Man uses fog to dart around the board and strike without warning. Sherlock, with the trusty Watson by his side, schemes and calculates to ensure victory. Dracula and the sisters slowly drain you of your power. Jekyll & Hyde uses the former's cunning and the latter's brute streng... | 0.966 | yes | |
| 6 | Unmatched: Little Red Riding Hood vs. Beowulf (2020) | In battle, there are no equals. ONCE UPON A TIME, Little Red Riding Hood took her basket of nasty tricks and faced off against the legendary Beowulf in this exciting Unmatched set. "What big eyes you have, Wulfie!" "That’s called 'rage', kid!" Little Red features a clever card-combo mechanism. Matching icons on the cards she plays to the one in her "basket" (discard pile), triggers potent e... | 0.963 | yes | |
| 7 | Unmatched: Battle of Legends, Volume One (2019) | In battle, there are no equals. Unmatched is a highly asymmetrical miniature fighting game for two or four players. Each hero is represented by a unique deck designed to evoke their style and legend. Tactical movement and no-luck combat resolution create a unique play experience that rewards expertise, but just when you've mastered one set, new heroes arrive to provide all new match-ups. Batt... | 0.954 | yes | |
| 8 | Unmatched: Robin Hood vs. Bigfoot (2019) | In battle, there are no equals. Unmatched is a highly asymmetrical miniature fighting game for two or four players. (Note that this set is solely for two players, but it can be combined with other sets, which all serve up to four players.) Each hero is represented by a unique deck designed to evoke their style and legend. Tactical movement and no-luck combat resolution create a unique play exp... | 0.930 | yes | |
| 9 | Cosmic Encounter Duel (2020) | The Cosmic Citizenship Council has announced it will allow two new alien species to join its ranks, but they forgot to make two copies of the filing form — which means that only one species can join! Now, the two candidates must battle for control of the planets to determine who deserves the right to become a Certified Civilization. Cosmic Encounter Duel is a competitive standalone two-player ... | 0.917 | no | |
| 10 | Marvel Champions: The Card Game (2019) | "With great power, there must also come great responsibility." –Stan Lee, Amazing Fantasy #15 Iron Man and Black Panther team up to stop Rhino from rampaging through the streets of New York. Captain Marvel and Spider-Man battle Ultron as he threatens global annihilation. Do you have what it takes to join the ranks of these legendary heroes and become a champion? Jump into the Marvel Univer... | 0.909 | yes | |
I’ll plot the top 10 games most likely to be owned by the user in the last 10 years of the training set.
Games highlighted in blue are currently in the user’s collection; games highlighted in light blue are games that the user previously owned.
| Top Games by Year for GOBBluth89 | |||||||||||
| Rankings based on predictive model trained on user's collection using games released through 2021 | |||||||||||
| Rank | 2011 | 2012 | 2013 | 2014 | 2015 | 2016 | 2017 | 2018 | 2019 | 2020 | 2021 |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | Dungeon Fighter | Merchant of Venus (Second Edition) | Impulse | Warhammer 40,000: Conquest | Forbidden Stars | Star Wars: Rebellion | Twilight Imperium: Fourth Edition | Concordia Venus | Unmatched: Battle of Legends, Volume One | Unmatched: Jurassic Park – InGen vs Raptors | Unmatched: Battle of Legends, Volume Two |
| 2 | A Few Acres of Snow | Star Wars: X-Wing Miniatures Game | Space Cadets: Dice Duel | Star Wars: Imperial Assault | A Game of Thrones: The Card Game (Second Edition) | Arkham Horror: The Card Game | Fallout | Newton | Unmatched: Robin Hood vs. Bigfoot | Unmatched: Cobble & Fog | Arkham Horror: The Card Game (Revised Edition) |
| 3 | Panic Station | Star Wars: The Card Game | BANG! The Dice Game | Star Wars: Empire vs. Rebellion | Fury of Dracula (Third/Fourth Edition) | Game of Thrones: The Iron Throne | Spirit Island | Sprawlopolis | Marvel Champions: The Card Game | Unmatched: Little Red Riding Hood vs. Beowulf | Great Western Trail (Second Edition) |
| 4 | Black Gold | Robinson Crusoe: Adventures on the Cursed Island | Lewis & Clark: The Expedition | Pandemic: The Cure | Mission: Red Planet (Second Edition) | DOOM: The Board Game | Legend of the Five Rings: The Card Game | Star Wars: X-Wing (Second Edition) | Star Wars: Outer Rim | Cosmic Encounter Duel | Cascadia |
| 5 | Hull Breach! | Android: Netrunner | Animals Frightening Night! | Camel Up | Drakon (Fourth Edition) | New Angeles | Century: Spice Road | Blackout: Hong Kong | KeyForge: Age of Ascension | Unmatched: Buffy the Vampire Slayer | Excavation Earth |
| 6 | Tinco | Cockroach Poker Royal | Crossing | Spyfall | XCOM: The Board Game | Love Letter: Premium Edition | Tournament Fishing: The Deckbuilding Game | Nemesis | The Crew: The Quest for Planet Nine | Calico | Descent: Legends of the Dark |
| 7 | The Lord of the Rings: The Card Game | Antike Duellum | Concept | Imperial Settlers | The King Is Dead | Codenames: Pictures | Stop Thief! | Root | Tiny Towns | Infinity Gauntlet: A Love Letter Game | For the King (and Me) |
| 8 | Rory's Story Cubes: Voyages | Terra Mystica | Relic | Steam Donkey | Blood Rage | Mansions of Madness: Second Edition | Gloomhaven | Arkham Horror (Third Edition) | Blitzkrieg!: World War Two in 20 Minutes | Fallout Shelter: The Board Game | The Crew: Mission Deep Sea |
| 9 | Keep Up The Fire!: The Boxer Rebellion | Fortress America | A Study in Emerald | The Werewolves of Miller's Hollow: The Pact | Runebound (Third Edition) | Kanagawa | Sheep 'n' Sheep | Star Wars: Legion | Era: Medieval Age | The Pet Cemetery | 1979: Revolution in Iran |
| 10 | Food Fight | Rumble in the Dungeon | Glass Road | Carcassonne Big Box 5 | Love Letter: Adventure Time | Codenames: Deep Undercover | Cat Lady | Everdell | Slyville | The March of Progress | Space Race |
The following table shows the model’s predictions for games in the training set.
What do the model’s predicted probabilties mean? Or, put another way, how well calibrated are the model’s predictions?
If the model assigns a probability of 5%, how often does the outcome actually occur? A well calibrated model is one in which the predicted probabilities reflect the probabilities we would observe in the actual data. We can assess the calibration of a model by grouping its predictions into bins and assessing how often we observe the outcome versus how often each model expects to observe the outcome.
A model that is well calibrated will closely follow the dashed line - its expected probabilities match that of the observed probabilities. A model that consistently underestimates the probability of the event will be over this dashed line, be a while a model that overestimates the probability will be under the dashed line.
I first assessed the models based on their performance via resampling on the training set.
But how well does my modeling approach do in predicting new games? To test this, I assessed the performance of the model (which was trained on games published through 2021) on games published in 2022-2023.
How well did the model do? The following table shows the model’s predictions for games in the validation set.
As before, I can then assess the performance of the model.
| type | wflow_id | .metric | .estimate |
|---|---|---|---|
| valid | glmnet | mn_log_loss | 0.008 |
| valid | lightgbm | mn_log_loss | 0.005 |
| valid | glmnet | roc_auc | 0.916 |
| valid | lightgbm | roc_auc | 0.983 |
What new and upcoming games does the model predict for GOBBluth89?
The following table displays the top 15 games published after 2021 with the highest probability of entering the user’s collection.
| Top 15 (Newer) Games for GOBBluth89 | |||||
| Rankings based on predictive model trained on user's collection using games released through 2021 | |||||
| rank | image | game | description | Pr(Own) | Own |
|---|---|---|---|---|---|
| 1 | Unmatched: Jurassic Park – Dr. Sattler vs. T. Rex (2022) | In battle, there are no equals. "Dinosaurs eat man… Woman inherits the earth." The greatest predator the world has ever known is closing in on the tenacious Dr. Sattler. Who has the slightest idea what to expect? In Unmatched: Jurassic Park – Dr. Sattler vs. T. Rex, the massive T rex unleashes fearsome attacks and seems unstoppable while Dr. Sattler makes full use of her surroundings and the a... | 0.959 | yes | |
| 2 | The Witcher: Old World (2023) | In The Witcher: Old World, you become a witcher — a professional monster slayer — and immerse yourself in the legendary universe of The Witcher franchise. Set years before the saga of Geralt of Rivia, The Witcher: Old World explores a time when monsters roamed the Continent in greater numbers, creating a constant peril that required the attention of expertly trained monster slayers, known as w... | 0.936 | no | |
| 3 | Undaunted: Battle of Britain (2023) | The RAF faces the Luftwaffe in this two-player deck-building game of aerial WWII combat. Summer, 1940. The German war machine has rolled through the continent in less than a year, crushing all before it. Britain is all that stands in the way of German victory in Europe. Command the pilots of the battle-tested Luftwaffe who are looking to extinguish all resistance and pave the way to invasion, ... | 0.748 | no | |
| 4 | Gloomhaven: Second Edition (2024) | Gloomhaven: Second Edition is a revised and elevated version of the award-winning core game of Gloomhaven. This is the culmination of everything Isaac Childres and the growing Cephalofair Games team have learned since the initial release of Gloomhaven, including feedback from the community, playtesters, co-designers, and developers. The world, story, and challenging gameplay are all still the ... | 0.724 | no | |
| 5 | Unmatched: Brains and Brawn (2023) | Unmatched: Brains and Brawn, the fifth and final Unmatched Marvel set, features some of Marvel's hottest heroes: Spider-Man, Dr. Strange, and She-Hulk. Spidey swings around the battlefield, using his spider-sense to keep him safe. Dr. Strange has, well, the best card names in the game: Behold the Seven Suns of Cinnibus! And She-Hulk won't think twice about throwing the book — or whatever heavy ... | 0.719 | no | |
| 6 | Unmatched: Houdini vs. The Genie (2022) | Unmatched is a highly asymmetrical miniature fighting game for two or four players. Each hero is represented by a unique deck designed to evoke their style and legend. Tactical movement and no-luck combat resolution create a unique play experience that rewards expertise, but just when you've mastered one set, new heroes arrive to provide all new match-ups. Unmatched: Houdini vs. The Genie adds ... | 0.651 | no | |
| 7 | Unmatched: Redemption Row (2022) | Unmatched is a highly asymmetrical miniature fighting game for two or four players. Each hero is represented by a unique deck designed to evoke their style and legend. Tactical movement and no-luck combat resolution create a unique play experience that rewards expertise, but just when you've mastered one set, new heroes arrive to provide all new match-ups. Unmatched: Redemption Row features th... | 0.585 | no | |
| 8 | Unmatched: Hell's Kitchen (2022) | Unmatched is a highly asymmetrical miniature fighting game for two or four players. Each hero is represented by a unique deck designed to evoke their style and legend. Tactical movement and no-luck combat resolution create a unique play experience that rewards expertise, but just when you've mastered one set, new heroes arrive to provide all new match-ups. Unmatched: Hell's Kitchen features th... | 0.585 | no | |
| 9 | Unmatched: Teen Spirit (2023) | Unmatched is a highly asymmetrical miniature fighting game for two or four players. Each hero is represented by a unique deck designed to evoke their style and legend. Tactical movement and no-luck combat resolution create a unique play experience that rewards expertise, but just when you've mastered one set, new heroes arrive to provide all new match-ups. Unmatched: Teen Spirit features four ... | 0.585 | no | |
| 10 | Unmatched: For King and Country (2023) | Unmatched is a highly asymmetrical miniature fighting game for two or four players. Each hero is represented by a unique deck designed to evoke their style and legend. Tactical movement and no-luck combat resolution create a unique play experience that rewards expertise, but just when you've mastered one set, new heroes arrive to provide all new match-ups. Unmatched: For King and Country featu... | 0.585 | no | |
| 11 | The Lord of the Rings: The Card Game – Revised Core Set (2022) | Sometimes, in order to truly appreciate a tale, one must first go back to its beginning. Grand adventures and strong fellowships are important and wonderful, but the first step of any journey is just as important as the last. With that in mind, it’s time to return to the beginning of one of the most epic adventures of all… With increased contents and some quality-of-life improvements, this new... | 0.576 | no | |
| 12 | Bardsung (2022) | You begin with a single tile and a destination. What unfolds is a branching, labyrinthine dungeon of dangerous enemies, wandering monsters, challenging puzzles, and valuable treasure. Bardsung is a game of choosing your path as you seek to inspire the bard’s songs and live your legend. From your physical passage through the dungeon to your journey of building your character into a fabled hero,... | 0.565 | no | |
| 13 | Desamparados: Stalingrado (2022) | ("Forsaken: Stalingrad" by its name in Spanish) narrates the events around both German and Soviet military platoons, after being abandoned to their fate in the middle of the Russian winter. With few resources and crippling cold, willpower is their best weapon. Players take a Side based on the army they want to play with, choose a Squad Deck, set up their Side Boards, Item Cards, their respecti... | 0.543 | no | |
| 14 | Distilled (2023) | Distilled is a highly thematic strategy card game about crafting spirits in a distillery, with resource management and push-your-luck elements. In the game, you have inherited a distillery and are hoping to someday achieve the title of master distiller through purchasing goods, building up your distillery, and creating the world's most renowned spirits. Use cards to purchase new ingredients an... | 0.450 | no | |
| 15 | Undaunted: Stalingrad (2022) | Stalingrad, 1942. Before you awaits a grueling conflict in this cornerstone battleground. As the bullets and bombs tear the city asunder, only through wits and valor can you seize the cornerstone of the entire Eastern Front and change the course of history. A heavy burden rests on your shoulders. Every casualty suffered in battle will weaken your forces for the entire campaign. Every bomb blas... | 0.401 | no | |
Why did the model predict these games?
Finally, I can examine predictions for all newer and upcoming games.